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Abstract 

Product codes are widely used in data-storage, optical and wireless applications. Their analytical performance 
evaluation usually relies on the truncated union bound, which provides a low error rate approximation based on the 
minimum distance term only. In fact, the complete weight enumerator of most product codes remains unknown. In 
' ' paper, concatenated representations are introduced and applied to compute the complete average enumerators of 
arbitrary product codes over a field Fq. The split weight enumerators of some important constituent codes (Hamming, 
Reed-Solomon) are studied and used in the analysis. The average binary weight enumerators of Reed Solomon product 
codes are also derived. Numerical results showing the enumerator behavior are presented. By using the complete 
enumerators, Poltyrev bounds on the maximum likelihood performance, holding at both high and low error rates, are 
, finally shown and compared against truncated union bounds and simulation results. 
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I. Introduction 

Product codes were introduced by Elias jlj in 1954, who also proposed to decode them by iteratively 
(hard) decoding the component codes. With the invention of turbo codes P], soft iterative decoding 
techniques received wide attention |j3j : low complexity algorithms for turbo decoding of product codes 
were first introduced by Pyndiah in Other efficient algorithms were recently proposed in jjSj and 
in [HI. 

For product codes, an interesting issue for both theory and applications regards the analytical 
estimation of their maximum likelihood performance. Among other, this analytical approach allows 
to: (i) forecast the code performance without resorting to simulation; (ii) provide a benchmark for 
testing sub-optimal iterative decoding algorithms; (iii) establish the goodness of the code, determined 
by the distance from theoretical limits. 

The analytical performance evaluation of a maximum-likelihood decoder requires the knowledge of 
the code weight enumerator. Unfortunately, the complete enumerator is unknown for most families of 
product codes. In these years, some progress has been made in determining the first terms of product 
code weight enumerators. The multiplicity of low weight codewords for an arbitrary linear product 
code were computed by Tolhuizen |7.. (In our paper, these results will be extended to find the exact 
input output weight enumerators of low weight codewords.) 

Even if the first terms can be individuated, the exact determination of the complete weight enumer- 
ator is very hard for arbitrary product codes [7], |H]- By approximating the number of the remaining 
codewords by that of a normalized random code, upper bounds on the performance of binary product 
codes using the ubiquitous union bound were shown in P'. However, this approximation is not valid 
for all product codes. 

In this paper, we will consider the representation of a product codes as a concatenated scheme 
with interleaver, and we will derive the average input-output weight enumerator for linear product 
codes over a generic field Fq. When combined with the extended Tolhuizen's result, this will provide 
a complete approximated enumerator for the product code. We will show how it closely approximates 
the exact weight enumerator. 
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Mostafa El-Khamy is with the Department of Electrical Engineering, California Institute of Technology, Pasadena, CA 91125, 
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This research was supported by NSF grant no. CCF-0514881 and grants from Sony, Qualcomm, and the Lee Center for Advanced 
Networking. 



2 



Previous work in the literature (see for example ^Oj, and reference therein) focused on estimating 
the product code performance at low error rates via the truncated union bound, using the enumerator 
low-weight terms only. By using the complete approximate enumerator, it is possible to compute the 
Poltyrev bound which establish tight bounds on the maximum likelihood performance at both 
high and low error rates. 

The outline of the paper is as follows. In section |TT1 we introduce the basic notation and definitions. 
In section IIIIl we extend Tolhuizen results and derive the exact input-output weight enumerator for 
product code low-weight codewords. Product code representation as serial and parallel concatenated 
codes with interleavers are introduced in section IIV-AI Uniform interleavers on finite fields with 
arbitrary size are discussed in section IIV-BI The average weight enumerators of product codes are 
then computed in sections IIV-CI The merge with exact low-weight terms, and the discussion of the 
combined enumerator properties are performed in section 

The computation of product code average enumerators relies on the knowledge of the input-redundancy 
weight enumerators of the component codes. For this reason, we derive in section IVII closed form for- 
mulas for the enumerator functions of some linear codes commonly used in the construction of product 
codes: Hamming, extended Hamming, and Reed Solomon codes. We proceed in section I VI II to de- 
rive the average binary weight enumerators of Reed Solomon product codes defined on finite fields of 
characteristic two. 

To support our theory, we present some numerical results. Complete average enumerators are 
depicted and discussed in section IVlll-AI Analytical bounds on the maximum likelihood performance 
are shown at both high and low error rates, and compared against simulation results in section IVlll-BI 
Finally, we conclude the paper in section IIXI 

II. Preliminaries 

Let Fq be a finite field of q elements, and C a (ric, fee, dc) linear code over Fq with codeword length 
ric, information vector length kc and minimum Hamming distance dc- 

The weight enumerator (WE) of C, Ec{h), (also called multiplicity) is the number of codewords 
with Hamming weight h: 

Ec{h) = \{ceC:^{c) = h}\, 

where w(-) denotes the symbol Hamming weight. 

For a systematic code C, the input-redundancy weight enumerator (IRWE), Rc{w,p), is the 
number of codewords with information vector weight w, whose redundancy has weight p: 

Rc{w,p) = \{c= {i\p) e C : w(j) = w w(p) = 

If T = (ni,n2) is a partition of the n coordinates of the code into two sets of size rii and n2, the 
split weight enumerator {wi,W2) is number of codewords with Hamming weights Wi and W2 in the 
first and second partition, respectively. If T is an (fc, n — k) partition such the first set of cardinality 
k constitutes of the information symbol coordinates, R{wi^W2) = {wi,W2)- 

The input-output weight enumerator (lOWE), Oc{w,h), is the number of codewords whose 
Hamming weight is h, while their information vector has Hamming weight w: 

Oc{w, h) = \{c E C : w(i) = w w(c) = h}\, 

For a systematic code, 

Oc{w,h) = Rc{w,h-w) . (1) 

It is also straight forward that 

Ecih) = Y,Oc{w,h) . (2) 

w=0 
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The WE function of C is defined by this polynomial in invariant Y: 

h=0 

while the IRWE function and the I OWE function of C are defined by these bivariate polynomials in 
invariants X and Y: 

f^c '^c 

Rc(X,F) = ^ ^ i?c(«^,p)X'"n (3) 

ui=0 p=0 

Oc(X,F) = ^^Oc(«;,/^)X"'F^ (4) 

w=0 h=0 

These functions are related by 

Oc{X,Y)=Rc{XY,Y) (5) 

and 

Ec{Y)=Rc{Y,Y) = Oc{l,Y). (6) 

In the following, we will denote the coefficient of X'^Y^ in a bivariate polynomial /(X, Y) by 
the coefficient function A{f{X,Y),X'"Y''). For example, Oc{w,h) = A(Oc(X, F), X"'F'^). Similarly, 
A (0(X, y), y"") is the coefficient of Y"^ in the bivariate polynomial 0{X,Y) and is a univariate 
polynomial in X. 

Let the code C be transmitted by a 2-PSK constellation over an Additive White Gaussian Noise 
(AWGN) channel with a signal-to- noise ratio (SNR) 7. The symbols $c(7) and $6(7) will denote the 
corresponding codeword error probability (CEP) and bit error probability (BEP) of a maximum like- 
lihood (ML) decoder, respectively. These ML performance can be estimated by computing analytical 
bounds based on the code enumerators. 

The truncated union bound, taking into account the minimum distance term only, provides a heuris- 
tic approximation commonly used at high SNR/low CEP: 

$,(7) ~ i Ec{d,) erfc W — 47 • (7) 

2 y He 

This formula provides a simple way for predicting the code performance at very high SNR, where 
maximum likelihood error events are mostly due to received noisy vectors lying in the decoding regions 
of codewords nearest to the transmitted one. Anyway, it is not useful in predicting the performance 
at low SNR. 

Tight bounds on the maximum likelihood codeword error probability of binary linear codes for 
AWGN and binary symmetric channel (BSC), holding at both low and high SNR, were derived by 
Poltyrev in ^I]. Other bounds such as the Divsalar simple bound and the variations on the Gallager 
bounds are also tight for AWGN and fading channels [12], [IH]- These bounds usually require knowledge 
of the complete weight enumerator Ec{h). In this paper, we will apply the Poltyrev bound by using a 
complete approximate weight enumerator of the considered product codes. 

Given the codeword error probability, the computation of the bit error probability may pose a 
number of technical problems. Let ^c{Ec{h),'y) denote the CEP over a channel with an SNR 7 
computed by using the weight enumerator Ec{h). The bit error probability ^b{l) is derived from the 
CEP by computing $^(7) = <l>,(/c(/i), 7), where Ic{h) = Et=i f 0(u;, h). 
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A common approximation in the literature is Ic{h) ~ -^^Ecih). This approximation is useful if 
the lOWE ■) is not known but the weight enumerator WE E{-) is. Some codes satisfy this 
approximation with equality: they are said to possess the multiplicity property. This is the case, 
for example, of all codes with transitive automorphism groups (including Hamming and extended 
Hamming codes) ^U] or all maximum distance separable codes (including Reed-Solomon codes) [T^ . 

Let IZ and C be (n,., dr) and (nc, fee, dc) linear codes over Fg, respectively. The product code 
whose component codes are TZ and C, V = TZ x C, consists of all matrices such that each row is a 
codeword in TZ and each column is a codeword in C. V is an {rip, kp, dp) linear code, with parameters 



III. Exact lOWE of Product Codes for Low Weight Codewords 

In [7j, Tolhuizen showed that in a linear product code V = TZ x C the number of codewords with 
symbol Hamming weight 1 < h < ho is: 

Ev{h) = ^y^Ec{T)En{h/i), (8) 
— 1 ^—^ 



where, given 
the weight ho is 



i\h 



w{dr, dc) = drdc + max(dr \—~\ , dc\—~\), 

q q 



^ _ i '^{drydc) + I, if g = 2 and both dr and dc are odd , . 

° w{dr,dc), otherwise ^ ' 

In particular, the minimum distance multiplicity of a product code is given by 

= En{dr)Ec{dc) _ 
q- 1 

These results are based on the properties of obvious (or rank-one) codewords of V, i.e., direct product 
of a row and a column codeword |7]. Let r & TZ and c G C, then an obvious codeword, p G P, is 
defined as 

Pi,j = riCj, (11) 

where is the symbol in the i-th coordinate of r and Cj is the symbol in the j-th coordinate of c. It 
follows that the rank of the x matrix defined by p is one and the Hamming weight of p is clearly 
the product of the Hamming weights of the component codewords, i.e., 

w(p) = w(r-)w(c). (12) 

Tolhuizen showed that any codewod with weight smaller than w{dr,dc) is obvious (Theorem 1, 7J) 
(smaller or equal if g = 2 and both dr and d^ are odd (Theorem 2, [7j)). The term in (jHJ and (fTUj) 
is due to the fact (Arj)(cj/A) are equal for all nonzero A G Eg. 

A generalization of Tolhuizen's result to input output weight enumerators is given in the following 
theorem. 

Theorem 1 

Let TZ and C be (n^, kr, dr) and (n^, kc, dc) linear codes over Eg, respectively. Given the product code 
= TZ X C, the exact lOWE for codewords with output Hamming weight 1 < h < ho is given by 

Or{w,h) = ^J2J20n{^,j)Oc{w/z,h/j), (13) 

i\w j\h 
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where the sum extends over all factors i and j of w and h respectively, and ho is given by © • V 
Proof: Let p G P be a rank-one codeword, then there exists a codeword r G 7^ and a codeword 
c G C such that p^j = ViCj. The krkc submatrix of information symbols in p could be constructed 
from the information symbols in c and r by for 1 < i < k^. and 1 < j < k^. It thus follows that 
the input weight of p is the product of the input weights of c and r while its output (total) weight is 
given by ()12j). Since all codewords with weights h < ho, are rank-one codewords, the theorem follows. 

■ 

These results show that both the weight enumerators and the input-output weight enumerators of 
product code low-weight codewords are determined by the constituent code low-weight enumerators. 
This is not the case for larger weights, where the enumerators of V are not completely determined by 
the enumerators of TZ and C jZj . 

It is important to note the number of rank-one low-weight codewords is very small, as shown by the 
following corollary regarding Reed Solomon (RS) product codes. 

Corollary 1 

Let C be an (n, k, d) Reed Solomon code over Fq. The weight enumerator of the product code V = CxC 
has the following properties, 

r 1, h = 0; 

Ev{h) = { {q-l){(:))\ h = <P- (14) 
[ 0, (f <h< d{d+l). 

V 

Proof: Let us apply (jHl). From the maximum distance separable (MDS) property of RS codes, 
d = n — k + 1 and n < q. It follows that w{d, d) = d{d + 1). Also Ec{d) = (q — 1) Q) . The first obvious 
codeword of nonzero weight has weight d"^. The next possible nonzero obvious weight is d{d + 1) which 
is w{d, d). ■ 

Example 1 

Let us consider the C(7, 5,3) RS code. The number of codewords of minimum weight is Ec{d) = 245. 
The complete lOWE function of C is equal to (this will be discussed in more detail in section IVI]) : 

Oc(X, Y) = 1+ 35XY^ + 140X^2 + 70X^Y^ + 350XV^ + TOOX^F^ + 175X^F^ 

+2660X^^5 + 2660X^F^ + 266X^1"^ + giTOX^F^ + 3668X^^6 + 12873X^y^ 

Let V be the square product code V = C x C. The minimum distance of V is dp = 9. By (jHj). 
its multiplicity is E-p{dp) = 8575. By applying Theorem [H the input-output weight enumerator for 
codewords in V with output weight dp = 9 is given by 

A(Op(X, r), Y^) = 175X + 1400X2 + 700X3 ^ 2800X^ + 2800X^ + 700X9. ^^^^ 

By Corollary n there are no codewords in V with either weight 10 or 11. No information is available 
for larger codeword weights 12 < t;; < 49. 

o 

The following theorem shows that rank-one codewords of a product code maintain the multiplicity 
property. 

Theorem 2 

If the codes C and 71 have the multiplicity property and V = IZ x C is their product code, then the 
subcode constituting of the rank-one codewords in V has this property. V 
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Proof: It follows from Th. Qtliat, for h < ho 



which proves the assertion. 

IV 

In the previous section, we have shown how to exactly compute the product code lOWE, for low 
weight codewords. For higher codeword weights, it is very hard to find the exact enumerators for an 
arbitrary product code over Fg. 

In this section, we will relax the problem of finding the exact enumerators, and we will focus on the 
computation of average weight enumerators over an ensemble of proper concatenated schemes. To do 
this: 

1. We will represent a product code as a concatenated scheme with a row- by-column interleaver. Two 
representations will be introduced. The first one is the typical serial interpretation of a product code, 
while the second one is a less usual parallel construction. 

2. We will replace the row-by-column interleavers of the schemes by uniform interleavers ^3], acting as 
the average of all possible interleavers. To do this, we will introduce and discuss uniform interleavers 
for codes over Fg. 

3. We will compute the average enumerator for these concatenated schemes, which coincide with the 
scheme enumerators if random interleavers were used instead of row-by-column ones. 

A code constructed using a random interleaver is no longer a rectangular product code. However, 
as we shall see, the average weight enumerator gives a very good approximation of the exact weight 
enumerator of the product code. This will confirm the experimental results by Hagenauer et al. that 
the error performance of linear product codes did not differ much if the row-column interleaver is 
replaced with a random interleaver (21 Sec. IV B]. We also confirm that numerically in section IVIIII 

A. Representing a Product Code as a Concatenated Code 

Let us first study the representation of a product code as a concatenated scheme with a row-by- 
column interleaver. 
Construction 1 

Given the code TZ^ric, kc, dc) , the augmented code 71^" is obtained by independently appending 
codewords of 71. The code 71^" has codeword length kcUr and dimension krkc- Moreover, its lOWE 
function is given by 

0'ciX,Y) = Oc.{X,Y) = (Oc(X,V))\ (16) 

See Fig. ^ The encoding process may be viewed as if we are first generating a codeword of TZ'^", 
with length kcUr symbols. The symbols of this codeword are read into an kc x rij. matrix by rows and 
read out column by column. In other words, the symbols of the augmented codeword are interleaved 



^ w=l '"^'"'^ i\w j\h 

\ - V ^ z ^ , , \ ^ if: 



^ j\h i=l t=l 

^^E^^(^')^c(Vj) 

q-l TlrTlc ^ 

-EAh), 



Average lOWE of Product Codes 
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by a row-by- column interleaver. Each column is then encoded into a codeword in C. The augmented 
columns form a codeword in V of length n^-nc. 

Observation 1: An {nrric, krkc, drdc) product code V = TZ x C is the serial concatenation of an 
[kcUr, kckj.) outer code TZ''" with an {ricrir, k^nr) inner code C"''" through a row-by-column interleaver 
TT with length = kcUr- (Equivalently V = TZ x C is the serial concatenation of an {krUc, krkc) outer 
code C'^'' with an {ncUr, k^nc) inner code TZ""" through a row- by-column interleaver with length k^Uc 
respectively.) 
Construction 2 

As an alternative, let the coordinates of a systematic product code be partitioned into four sets as 
shown in Fig. |21 We can introduce the following parallel representation. 

Observation 2: See Figure |21 An {nrUc, krkc, drdc) product code can be constructed as follows: 

1. Parallel concatenate the (urkckrkc) code TZ'^'^, with the {uckr, kckr) code C^'' through a row-by- 
column interleaver tti of length A^i = krkc- 

2. Interleave the parity symbols generated by TZ'^'' with a row-by-column interleaver 112 of length N2 = 

3. Serially concatenate these interleaved parity symbols with the {nc{nr — kr), kc{nr — kr)) code C^-^-fcr^ 

B. Uniform Interleavers over Fq 

Given the two product code representations just introduced, we would like to substitute the row- 
by-column interleavers with uniform interleavers. In this section, we then investigate the uniform 
interleaver properties, when the interleaver is a symbol based interleaver and the symbols are in Fg. 
The concept of uniform interleaver was introduced in [15j and ^H] for binary vectors in order to study 
turbo codes: it is a probabilistic object acting as the average of all possible interleavers of the given 
length. In the binary case, the number of possible permutations of a vector of length L and Hamming 
weight w is (^) . Let us denote by V{L, w) the probability that a specific vector is output by the 
interleaver when a vector of length L and input w is randomly interleaved. In this binary case we have 

V{L,w) = ^. (17) 

If t; is a vector on Fq of length L and the frequency of occurrence of the q symbols is given by 
/o,/i, ...,lq-i respectively, then the number of possible permutations is given by the multinomial coef- 
ficient j1 7j 

L! 

However, this requires the knowledge of the occurrence multiplicity of each of the q symbols in the 
permuted vector. 

We then introduce here the notion of uniform codeword selector (UCS). Let us suppose a specific 
vector of symbol weight w and length L is output from the interleaver corresponding to a certain 
interleaver input with the same weight. This vector is encoded by an (A^, L) code C following the 
interleaver. 

We assume that all the codewords of C with input weight w have equal probability of being chosen 
at the encoder's output. The UCS picks one of these codewords (with input weight w) at random. 
Thus the probability that a specific codeword is chosen by the UCS is 

V(L,w) = ^ - = ^-^7T, (18) 



where Ylh^c{w, h) is the total number of codewords with input weight w. This is equivalent to a 
uniform interleaver over Fg which identifies codewords by their Hamming weights. It is noticed that 
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for the binary case, the uniform interleaver (|T7jl is equivalent to the UCS |T8|) . The UCS has the 
property of preserving the cardinahty of the resuhing concatenated code. 

C. Computing the Average Enumerators 
Construction 1 

Given the Construction 1 of Obs. ^ and Fig. ^ let us replace the row-by-column interleaver vr of 
length N = kji^ with a uniform interleaver over Fq of the same length. It is easy to show that the 
average lOWE function of the product code V is given by 

Qv{X,Y) = ^V{kcnr,w)K{Q>^Ti{X,Y),Y'")K{Ql^\X,Y),X'"). (19) 

«)=0 

The average weight enumerator function Ep(y) can be computed from 0-p(X, F) by applying (jHl). 
Construction 2 

Given the Construction 2 of Obs. |21and Fig. IHl let us replace the two row-by-column interleavers tti 
of length A^^i = krkc and 112 of length N2 = kc{nr — kr), with two uniform interleaver overs Fq of length 
Ni and N2, respectively. 

We begin by finding the partition weight enumerator (PWE) of the code V resulting from the parallel 
concatenation of TZ^" with C'^''. We have: 

Fi){W,X,Y) = ^\/(A;,/ce,u;)A (M^^liy,^),!^") A ly", (20) 

w=Q 

where PTi{w,x,y) is the number of codewords in the parallel concatenated code with weights w, x 
and y in the partitions constituting of information symbols, checks on rows and checks on columns 
respectively, and is given by 

kckr fcc(nr-fcr) kr{nc-kc) 

¥j,{W,X,Y) = Y, E E Pv{w,x,y)W'"X''Yy. (21) 

(Note that M.d{W, X) = Px>(PV, X, X) gives the average IRWE function of a punctured product code 
with the checks on checks deleted.) 

The partition weight enumerator function of the product code V is then given by 

Ft,{W,X,Y,Z) = J2 V{k^{nr-kr),x)A{R'^'-'''-{X,Z),X'')A{FTy{W,X,Y),X'')X''. (22) 

x=0 

The PWE, P-p{w, X, y, z), enumerates the codewords with a weight profile shown in Fig. |21and is given 
by expanding the PWE function F-p{W, X, Y, Z) as follows, 

kckr kc{nr~kr) kr{nc-kc) {nr-kr){nc-kc) 

Mw,x,Y,z) = Y, E E E My^,^,y,^)w'"x''y'z\ (23) 

w=o x=o y=o 2=0 

It follows that the average IRWE function of V is Mp(X,F) = Pp(X, F, y, F). Consequently, the 
lOWE function O-p^X, Y) can be obtained via (jSj) and the weight enumerator function ]Ep(y) via ©. 
By using (fTHj). the cardinality of the code given by E-p{Y) is preserved to be 
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V. Merging Exact and Average Enumerators into Combined Enumerators 

The results in the previous section are now combined with those of section UTTl reflecting our knowl- 
edge of the exact lOWE of product codes for low weights. Let ho be defined as in (jHI). We introduce 
a complete lOWE which is equal to: 

• the exact 10 WE for h < ho] 

• the average lOWE for h > ho'- 

dp(X, Y) = Y,Y. h)X'"Y'^, (24) 



w=0 h=0 



such that 



where 0-p{w, h) is given by Th. ^ while 0'p{w, h) = A(Op(X, Y), X^Y^) is derived as in section HV-CI 
We will call 0-p{X,Y) the combined input output weight enumerator (CIOWE) of V. The 

corresponding combined weight enumerator function K-p{Y) can be computed by ©. 

Let us now discuss some properties of the CIOWE. Let W{C) = {h : Ec{h) ^ 0} be the set of 
weights h, such that there exists at least one codeword c G C with weight h. Observe that the weight 
of a product codeword p E V is simultaneously equal to the sum of the row weights and to the sum of 
the column weights. We define an integer h a plausible weight of p E V, if h could be simultaneously 
partitioned into Uc integers restricted to W(JV) and into Ur integers restricted to W{C). 

Note however, that not all plausible weights are necessarily in W{V). 

Theorem 3 

Suppose V = TZ X 71, (the row code 71 is the same as the column code C), then the set of weights 
with a non-zero coefficient in the average weight enumerator of V derived by either ()19j) or (j22I) are 
plausible weights for the product code. V 
Proof: The set of plausible weights of a product code is the set of weights h such the coefficients 
of y'^ in both (Ec(F))""' and {KulY))^" is non-zero. When 7^ = C, it suffices to show that for any 
weight h if the coefficient of Y'^ in K'p{Y) is non-zero, then it is also non-zero in (Ec(y))"''. 

For Construction 1, let ¥^{Y) be the average weight enumerator derived from ()19p by ]Ep(y) = 
Op(l,F). Since all output weights that appear in Op(l,y) are obtained from A{0^'-{X,Y),X'") 
then, by ()16p . they have nonzero coefficients in (Ec(y))""' and we are done. 

For Construction 2, let K-p(Y) be the average weight enumerator derived from (|22j) by E,-p{Y) = 
Pp(y,y,F,F). Let T{W,Y) = A{Ft,{W,X,Y),X^). From ((201), it follows that any exponent with 
a nonzero coefficients in T(Y,Y) also has a non-zero coefficient in R^'' (Y,Y) or equivalently E^'-(F). 
Similarly if T'(X,Z) = A (]R^''^^''(X, Z), X^) X^, then any exponent with a non-zero coefficient in 
T'(Y,Y) also has a non-zero exponent in E^' "'^''. It follows from (j22j) that any exponent with a 
non-zero coefficient in ]Ep(y) also has a non-zero coefficient in E^''~'^''E^'' and we are done. ■ 
In 10], the authors approximated the weight enumerator of the product code by a binomial distribution 
for all weights greater than ho- Our approach has the advantage that only plausible weights appear in 
the combined enumerators of the product code. 

VI. Split weight enumerators 

As seen in the previous section, deriving the CIOWE of the product code requires the knowledge of 
the IRWE of the component codes. In this section we discuss the weight enumerators of some codes 
which are typically used for product code schemes. In particular, we show closed form formulas for 
the IRWE of Hamming, extended Hamming, Reed Solomon codes. To do this, it is sometimes easier 
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to work with the spht weight enumerator (SWE, see definition in section |n} of the dual code. The 
connection between the IRWE of a code and its dual was established in fB]- The following theorem 
gives a simplified Mc Williams identity relating the SWE of a linear code with that of its dual code in 
terms of Krawtchouk polynomials. 

Theorem 4 

Let C be an (n, k) linear code over Fg and be its dual code. Let A{a, (3) and A-*- (a, /3) be the SWEs 
of C and C"*" respectively for an (rii, 77-2) partition of their coordinates, then 

^ ni n2 

A^{a,j3) = -—r^^A{w,v)}Caiw,ni)]Cf3{v,n2), 

' ' w=0 v=0 

such that for /5 = 0, 1, ...,7, lCp{v,'-)) = J2j=o (/3^P (j) (O' ~ l)'^"-' is the Krawtchouk polynomial. 

V 

Proof: By a straight forward manipulation of the Macwilliams identity for the split weight 
enumerator Ch. 5, Eq. 52] [20], it follows that for linear codes and r = q — 1, 

= — ^^A(w,t;)(l-rX)"^'"'(l -X)™(1 -rr)"2-^(l (26) 

' w=0 v=0 

where A{X,Y) and A^{X,Y) are the SWE functions of C and C"*" respectively. Observing that for a 
positive integer 7 and < /3 < 7, (1 — rF)'^~''(l — ¥)"" = X]/3=o ^/sl"^' t)^'^ is the generating function 
for the Krawtchouk polynomial fHl Ch. 5, Eq. 53] and that A-^{a,P) is the coefficient of X°'Y^ in 
the right-hand side of the result follows. ■ 
By observing that the roles of the input and the redundancy are interchanged in the code and its 
dual, we have: 

Corollary 2 

The IRWEs of C and are related by 

^ n—k k 

R^{a, P) = T^\Y.Y. ^(^' k)IC4v, n-k). 

' ' v=0 w=0 

V 

A. Hamming Codes 

The IRWE function of systematic Hamming codes could be derived by observing that they are the 
dual code of simplex codes JH]) [21] ■ A recursive equation for evaluating the IRWE of Hamming 
codes was given in [22] • The following theorem gives a closed form formula for the IRWE function of 
Hamming codes in terms of Krawtchouk polynomials. 

Theorem 5 

The IRWE of (2™ — 1, 2"* — m — 1, 3) (systematic) Hamming codes is 

Rnia, /?) = ^ /C,(^, m)/C„(2-- -^v,2--m-l)+ ('"^ " ^) ) . 

V 

Proof: By observing that the IRWEF of the (2™ - l,m,2"'-^) simplex code is Ms(X,y) = 
1 + J2w=i {'^)X'^Y^"'~'-'^. Using Cor. |2and observing that /C^(0,m) = ("), we obtain the result. ■ 
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B. Extended Hamming Codes 

Extended Hamming codes were studied in JU], where it was shown they possess the multiphcity 
property, and closed-form formulas for their input-output multiplicity were provided. The following 
theorem shows a closed expression for their IRWE function in terms of Krawtchouk polynomials. 

Theorem 6 

A closed form formula for the IRWE of the (2™, 2™ — m — 1, 4) Extended Hamming codes is 
REHia, (3) = ^\ y:r^^) ICpiw, m + 1) /C«(2'"-i -w,2^-m-l) 



1 ( fr 

\w=l ^ 



W 



_v 

Proof: By observing that the extended Hamming codes are the duals of the (2'",m + 1,2™^^) 
first order Reed Muller (RM) codes whose IRWE function could be shown to be ]R(X, Y) = 1 + 
^^+i^2--m-i ^ ^m^^ ^m+i^^ay2— i-a^ |2] ^nd obscrviug that ]Cp{-f,-f) = ©(-1)^ the 

result follows. ■ 
Note that the WE of extended Hamming (EH) codes could also be derived from that of Hamming 
(H) codes by using the well known relation [1^], EEH{h) = Enih) + Eff{h — 1) if /i is even and is zero 
otherwise. It follows that 

REH{a,(3) = ^^^ otherwise ^^^^ 

C. Reed Solomon Codes 

Reed Solomon codes are maximum distance separable (MDS) codes [T^. The SWE of MDS codes 
was recently studied by El-Khamy and McEliece in where this theorem was proved: 

Theorem 7 

HH The SWE of MDS codes is given by 



( n y 

V 



It follows that the IRWE of an (n, k) systematic RS code is given by: 

RRsicy,/3) = E{a + /3)- 



VII. Average Binary IRWE of product Reed Solomon Codes 

Recently, new techniques for decoding Reed Solomon codes beyond half the minimum distance were 
derived in and algebraic soft decision algorithms were proposed (see and references therein). 
In this section we derive a number of results on RS product codes and their binary image. 

Let d{X) = Yli=o diX^ be a data polynomial over Eg. Then an (n, k) Reed Solomon code is generated 
by evaluating the data polynomial d{X) at n distinct elements of the field forming a set called the sup- 
port set of the code S = {ao, ai, On-i}- The generated codeword is c = {d{ao), d{ai), d{an-i)}- 
The resulting code is not systematic. In the following theorem we show how a product of two RS codes 
can be generated by polynomial evaluation of a bivariate polynomial. 
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Theorem 8 

Let the krkc data symbols dij be given by the bivariate polynomial D(X, Y) = Yli=o^ X]j=o^ dijX^Y^ . 
Let the support set of the row and column codes, TZ and C respectively, be given by 5"^ = {ao, ai, dnr-i} 
and Sc = {Po,Pi, ■■■yPn^-i} respectively. Then the product code V = 71 x C is generated by Vij = 
D{ai, Pj) for i = 0, ..,nr — 1 and j = 0, nc — 1. V 
Proof: To prove that V is really the product of TZ and C, we prove that each row, r, P,, = 
{Vij : i = {0, rir — = r} is a codeword in 71 and each column c, Pc,* = {7^i,j '■ 3 = {0, ric — 

= c} is a codeword in C. The r-th row is given by P^,* = {D{cio, Pr), /S^), -D(q;„^_i, /^r)}. 

Observe that = Eto' Et=o ^^.-("O'l/^^)" = Eto' (Et=o ^^,-(/?^)") ("O'^- Let = 

Ylw=o '^'^,wiPr)^ ^iid Y = {Y '■ = 0,...,kr — 1}, then Y forms the information vector which is 
encoded into the r-th row V^^^r = {D'io^o), D'{ai), Z)'(a„^_i)} by D'{X) = YliLo^ li-^^- This proves 
that is a codeword in 7Z. 

Similarly, any column c could be expressed as 7^c,* = {D" iPo) , D" iPi) , -D"(/5„^-i)}, 
where D"{X) = J2 -=o ^nd = ^ to 

J (etc)* is the j-th element in the information vector for 
column c. Thus each column is a codeword in C. 

Since the cardinality of this code is q^'-^^ we are done. ■ 
(This proof also gives insight how algebraic soft decision decoding can be used for iteratively decoding 
the component RS codes of the product code.) 

Given the product of Reed Solomon codes defined over a field of characteristic two, it is often the case 
that the binary image of the code is transmitted over a binary-input channel. The performance would 
thus depend on the binary weight enumerator of the component RS codes, which in turn depends on 
the basis used to represent the 2'"-ary symbols as bits. Furthermore, it is very hard to find the exact 
binary weight enumerator of a RS code for a specific basis representation [221 . The average binary 
image of a class of generalized RS codes has been studied in j^HI- The average binary image for codes, 
defined over finite fields of characteristic two, was derived by assuming a binomial distribution of the 
bits in the non-zero symbols in j27j. Let Cb denote the binary image of an (n, k) code C which is defined 
over the finite field F2m. Let E,c{Y) be the weight enumerator function of C. Then the average weight 
enumerator of the {nm, km) code Cb is given by [2Z] 

Ec,{Y)=Eci^iY)), (28) 

where "^(Y) = ^^^rziii^ + Y)"^ — 1) is the generating function of the bit distribution in a non-zero 
symbol. We assume that the distribution of the non-zero bits in a non-zero symbol follows a binomial 
distribution and that the non-zero symbols are independent. If the coordinates of the code C are 
split into p partitions, then there is a corresponding p-partition of the coordinates of Cb, where each 
partition in Cb is the binary image of a partition in C. By independently finding the binary image 
of each partition, the average partition weight enumerator of Cb could be derived as in the following 
lemma. 

Lemma 1 

[n|Let Fp{W, X, Y,Z) be the PWE function of a code V defined over Fg™. The average PWE of the 
binary image n is Pp,(iy, X, F, Z) = Pp (^(PF), ^(X), *(r), *(Z)) . V 

Corollary 3 

If Mp(X, Y) is the combined IRWE of the (n^, kp) product code P defined over 1 

then the combined 

IRWE of its binary image is 

Mp,(X,F)=ip(vl/(X),vI/(F)), 

where 

^ix) = ^J-^iii+xr-i) 
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and 

kpm npm—kpm 

V 

Note this same formula does not hold in the case of the lOWE. However, the binary lOWE could 
be derived from the binary IRWE by using 

VIII. Numerical Results 

In this section we show some numerical results supporting our theory. The combined input output 
enumerators of some product codes are investigated in section IVIII-AI Analytical bounds to ML 
performance are computed and discussed in section IVIII-BI Hamming codes, extended Hamming 
codes and Reed Solomon codes are considered as constituent codes. 

A. Combined Input Output Weight Enumerators 
Example 2 

Let us consider the extended Hamming code (8,4). From Th. IHl its lOWE function is 

OEHiX, Y) = l + AXY^ + QX^Y^ + AX^Y^ + X^Y^ . 

Let us now study the square product code (8, 4)^. By applying (fT^ we can derive the average weight 
enumerator function obtained with the serial concatenated representation. By rounding to the nearest 
integer, we obtain: 

Ev(Y) =1 +3F*^ + 27r^^ + 107y^^ + 604r^° + 3153r^^ + 13653F^^ + 30442r^^ + 
+13653r^*^ + 3153r^° + QOAY^^ + 107F^^ + 27Y^^ + 3r^^ + Y^^. 

By (I22|) . we can derive the average weight enumerator function obtained with the parallel concate- 
nated representation. We obtain: 

Ep(y) = 1 +2Y^ + 26y^2 ^ ggyie ^ 533^20 _^ 3115^24 _^ 137301-28 _^ 39353^32 ^ 

+ 13780F^^ + 3116F^° + 568^^^ + 98Y^^ + 26Y^^ + 2Y^^ + Y^^ (29) 

(For space limitations we do not show the lOWE functions.) Note that all codewords are of plausible 
weights as expected from Th. El It could be checked that in both cases, the cardinality of the code 
(without rounding) is preserved to be 2^^. In general, the parallel representation gives more accurate 
results than the serial one, and will be used for the remaining results in this paper. 

For low-weight codewords, we can compute the exact lOWE. By Th^ the exact lOWE of the 
product code for weights less than ho = 24 is equal to 

Or{X, y) = 1 + leXF^^ + 48X2yi6 + s2X^Y^^ + 36X^^1^ + ASX^Y^^ + IQX^Y^^. 
It follows that the combined weight enumerator function for this product code is 

E-piY) = 1 +196yi^ + 3116r=^^ + 1378iy28 + 30353F32 

+13781F^^ + 3116r^° + 568F^^ + 98^^^ + 26Y^^ + 2Y^^ + Y^^. 
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A symmetric weight enumerator of the component codes imphes a symmetric one for the product 
code. Thus, by the knowledge of the exact coefficients of exponents less than 24, K-p(Y) could be 
improved by setting the coefficients of y52 yse ^ero and adjusting the coefficients of 

the middle exponents such that the cardinality of the code is preserved. We obtain: 



EV(F) = 1 +196Y^^ + 3164^2^ + 13995^^^ + 30824^^2 

+13995y^^ + 3164r^° + 196^^^ + (30) 

In this case, the exact weight enumerator can be found by exhaustively generating the 65536 code- 
words of the product code, and it is equal to: 



Ep(r) = 1 +196y^^ + 4704^24 + 10752^28 + 342301^^^ 
+ 10752^3^ + 4704y^° + 196F^^ + F^^ 

It can be verified that the combined weight enumerator ()3()|1 gives a very good approximation of this 
exact weight enumerator. 

o 

Example 3 

The combined weight enumerator of the extended Hamming product code (16, 11)^, computed by 
applying ()2()|1 and (j22I), is depicted in Fig. 01 It is observed that for medium weights, the distribution 
is close to that of random codes, which is given by 

E{w) = q-^^"-^"^ (^^^ (g - 1)-, 
except that only plausible weights exist. 



Example 4 

The combined symbol weight enumerator of the (7, 5)^ product RS codes over Fg, computed by applying 
fj2U|) and ()22|) . is shown in Fig. |S| It can be observed that the weight enumerator approaches that of a 
random code over Fg for large weights. The average binary weight enumerator of the (147, 75) binary 
image, obtained by applying Corollary is shown in Fig. IHl It is superior to a random code at low 
weights and then, as expected, approaches that of a binary random code. o 

B. Maximum Likelihood Performance 

In this section, we investigate product code performance. The combined weight enumerators are 
used to compute the Poltyrev bound ^T], which gives tight analytical bounds to maximum likeli- 
hood performance at both high and low error rates. For proper comparison, truncated union bound 
approximation and simulation results are also considered. 

Example 5 

The codeword error rate (CER) and the bit error rate (BER) performance of two Hamming product 
codes ((7,4)^ and (31,26)^) are shown in Fig. [7| We have depicted: 

• The Poltyrev bounds on ML performance (P on the plots), obtained by using the combined weight 
enumerator computed via (j221)- 

• The truncated union bound (L on the plots), approximating the ML performance at low error rates, 
and computed from the minimum distance term via ((7j). 
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• The simulated performance of iterative decoding (S on the plots), corresponding to 15 iterations of 
the BCJR algorithm on the constituent codes trellises ([Hj.pTl]). 

By looking at the results, we can observe that: 

• The combined weight enumerators derived in this paper, in conjunction with the Poltyrev bound, 
provide very tight analytical bounds on the performance of maximum likelihood decoding also at low 
SNRs (where the truncated union bound does not provide useful information). 

• For the (7,4)^ code the exact enumerator can be exhaustively computed, and the exact Poltyrev 
bound is shown in the figure. It is essentially identical to the bound computed with the combined 
weight enumerator. 

• The ML analytical bounds provide very useful information also for iterative decoding performance. 
In fact, the penalty paid by iterative decoding with respect to ideal ML decoding is very limited, as 
shown in the figure (feedback coefficients for weighting the extrinsic information and improve iterative 
decoding has been employed, as explained in \10\). 

o 

Example 6 

The performance of the extended Hamming product code (32, 26)^ is investigated in Fig. |H1 Also in 
this case, the tightness of the bounds is demonstrated, for both the CER and the BER. With the aid 
of the Poltyrev bound for the BSC channel, hard ML bounds have also been plotted. It is shown that 
soft ML decoding on the AWGN channel offers more than 2 dB coding gain over hard ML decoding, o 

Example 7 

In Fig. El the performance of soft and hard ML decoding of various Hamming and extended Hamming 
codes are studied and compared. As expected, the EH product codes show better performance than 
Hamming product codes of the same length due to their larger minimum distance and lower rate. (For 
the (7,4)^ Hamming product code and the (8,4)^ extended Hamming product code, it is observed 
that the bounds using our combined weight enumerator overlapped with ones using the exact weight 
enumerators, which can be calculated exhaustively in these cases.) o 

Example 8 

The performance of the binary image of some Reed Solomon product codes, for both soft and hard 
decoding, are investigated in Fig. [TUl where the Poltyrev bound has been plotted. As expected, soft 
decoding has about 2 dB of gain over hard decoding. It can be observed that these product codes have 
good performance at very low error rates (BER lower than 10~^), where no error floor appears. 

It is well known that the sphere packing bound provides a lower bound to the performance achiev- 
able by a code with given code-rate and codeword length [2H]- The discrete-input further limitation 
occurring when using a given PSK modulation format was addressed in |29|. The distance of the code 
performance from this theoretical limit can be used an indicator of the code goodness. 

Let us consider, for example, the (15, 11)^ RS product code, corresponding to a (900,484) binary 
code. By looking at the Poltyrev bound plotted in Fig. ^[ this code achieves a BER=10~^° for a 
signal-to-noise ratio 7 ~ 2.2 dB. By computing the PSK sphere packing bound for this binary code, 
we obtain a value of about 1.9 dB for BER=10~^'^. This means that this RS product code is within 
0.3 dB from the theoretical limit, which is a very good result at these low error rates. 

o 

IX. Conclusions 

The average weight enumerators of product codes were studied in this paper. The problem was 
relaxed by considering proper concatenated representations, and assuming random interleavers over 
Fq instead of row-by-column interleavers. The exact lOWE for low-weight codewords were also derived 
by extending Tolhuizen results. By combining exact values and average values, a complete combined 
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weight enumerator was computed. This enables us to study the ML performance of product codes at 
both low and high SNRs by applying the Poltyrev bound. The computation of average enumerators 
requires knowledge of the constituent code enumerators. Closed form formulas for the input redun- 
dancy enumerators of some popular codes were shown. The binary weight enumerator of ensemble of 
product RS codes was also derived. 

The combined weight enumerators of Hamming and Reed Solomon product codes were numerically 
computed and discussed. Using the combined enumerators, tight bounds on the ML performance of 
product codes over AWGN channels were derived by using the Poltyrev bounds. The tightness of the 
bounds were demonstrated by comparing them to both truncated union bound approximations and 
simulation results. 

In particular, Reed Solomon product codes show excellent performance. Reed Solomon codes are 
widely used in wireless, data storage, and optical systems due to their burst-error correction capabil- 
ities. The presented techniques allow to analytically estimated Reed Solomon product codes perfor- 
mance, and show they are very promising as Shannon-approaching solutions down to very low error 
rates without error floors. This suggests the search for low-complexity soft decoding algorithms for 
Reed Solomon codes as a very important research area in the near future. 
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Fig. 1 

Construction 1: serial concatenation. 
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Fig. 2 

The four set partition of the coordinates of a product codeword used in Construction 2. 
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Construction 2: parallel concatenation. 
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Average weight enumerator of the (1 6,1 1 ) extended Hamming product code 
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Fig. 4 

The combined weight enumerator of the (16,11)^ extended Hamming product code is compared with 

THAT OF A random BINARY CODE OF THE SAME DIMENSION. 
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Average Weight Enumerator of the (7,5f RS Product Code 
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Fig. 5 

The combined symbol weight enumerator of the (7, 5)^ Reed Solomon product code is compared 

WITH that of a random CODE OVER Fg WITH THE SAME DIMENSION. 
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Fig. 6 

The combined binary weight enumerator of the binary image of the (7, 5)^ Reed Solomon product 

CODES IS compared WITH THAT OF A RANDOM BINARY CODE WITH THE SAME DIMENSION. 
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SNR (dB) 

Fig. 7 

CER AND BER PERFORMANCE OF SOME HAMMING PRODUCT CODES FOR SOFT DECODING OVER AWGN 
CHANNEL. The POLTYREV bound P, and THE TRUNCATED UNION BOUND APPROXIMATION L, ARE COMPARED TO 
SIMULATED PERFORMANCE OF ITERATIVE DECODING S. FOR THE (7,4)^ CODE, THE POLTYREV BOUND COMPUTED 

WITH THE EXACT WEIGHT ENUMERATOR IS ALSO REPORTED. 
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Fig. 8 

CER AND BER PERFORMANCE OF THE (32, 26)^ EXTENDED HAMMING PRODUCT CODE FOR SOFT AND HARD 
DECODING OVER AWGN CHANNEL. THE POLTYREV BOUND P AND THE TRUNCATED UNION BOUND 
APPROXIMATION L ARE COMPARED TO SIMULATED PERFORMANCE OF ITERATIVE DECODING S. 
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Performance of Hamming product codes 
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Fig. 9 

BER PERFORMANCE OF HAMMING PRODUCT CODES (UPPER) AND EXTENDED HAMMING PRODUCT CODES 
(lower) OVER AWGN CHANNEL FOR BOTH SOFT DECISION (SD) AND (HD) HARD DECISION. THE POLTYREV 
UPPER BOUND (UB) AND THE TRUNCATED UNION BOUND APPROXIMATION (LB) ARE USED FOR SD, WHILE THE 

POLTYREV BOUND FOR THE BSC IS USED FOR HD. 
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Fig. 10 

BER PERFORMANCE OF SOME REED SOLOMON PRODUCT CODES OVER THE AWGN CHANNEL FOR BOTH SOFT 
DECISION (SD) AND HARD DECISION (HD) DECODING, OBTAINED BY PLOTTING THE POLTYREV BOUND COMPUTED 

VIA THE COMBINED WEIGHT ENUMERATORS. 



